48 research outputs found
Discrete-time Analysis of Multicomponent GI/GI/1 Queueing Networks
In this work, we provide initial insights regarding the error introducedinto multicomponent queueing systems by assuming the departure processes of arbitraryGI/GI/1-oo queues to be renewal processes. To this end, we compute the sojourntime distribution as well as departure distributions of a linear chain of queueingcomponents and compare the results to a simulation of the same system. By applyingthe renewal approximation, potential autocorrelations of the departure processesare lost. We investigate the magnitude of this error regarding both the sojourn timeas well as interdeparture time distributions for a broad set of parameters. Althoughmore indepth studies are needed, our results show that both distributions can beclosely approximated, which allows the application of the model to asses the performanceof real world NFV function chains
Power Reduction Opportunities on End-User Devices in Quality-Steady Video Streaming
This paper uses a crowdsourced dataset of online video streaming sessions to
investigate opportunities to reduce the power consumption while considering
QoE. For this, we base our work on prior studies which model both the
end-user's QoE and the end-user device's power consumption with the help of
high-level video features such as the bitrate, the frame rate, and the
resolution. On top of existing research, which focused on reducing the power
consumption at the same QoE optimizing video parameters, we investigate
potential power savings by other means such as using a different playback
device, a different codec, or a predefined maximum quality level. We find that
based on the power consumption of the streaming sessions from the crowdsourcing
dataset, devices could save more than 55% of power if all participants adhere
to low-power settings.Comment: 4 pages, 3 figure
How do crowdworker communities and microtask markets influenceeach other? a data-driven study on amazon mechanical turk
Crowdworker online communities -- operating in fora like mTurkForum and TurkerNation -- are an important actor in microwork markets. Albeit central to market dynamics, how the behavior of crowdworker communities and the dynamics of online marketplaces influence each other is yet to be understood. To provide quantitative evidence of such influence, we performed an analysis on 6-years worth of mTurk market activities and community discussions in six fora. We investigated the nature of the relationships that exist between activities in fora, tasks published in mTurk, requesters for such tasks, and task completion speed. We validate -- and expand upon -- results from previous work by showing that (i) there are differences between market demand and community activities that are specific to fora and task types; (ii) the temporal progression of HIT availability in the market is predictive of the upcoming amount of crowdworker discussions, with significant differences across fora and discussion categories; (iii) activities in fora can have a significant positive impact on the completion speed of tasks available in the market
From QoS Distributions to QoE Distributions: a System's Perspective
In the context of QoE management, network and service providers commonly rely
on models that map system QoS conditions (e.g., system response time, paket
loss, etc.) to estimated end user QoE values. Observable QoS conditions in the
system may be assumed to follow a certain distribution, meaning that different
end users will experience different conditions. On the other hand, drawing from
the results of subjective user studies, we know that user diversity leads to
distributions of user scores for any given test conditions (in this case
referring to the QoS parameters of interest). Our previous studies have shown
that to correctly derive various QoE metrics (e.g., Mean Opinion Score (MOS),
quantiles, probability of users rating "good or better", etc.) in a system
under given conditions, there is a need to consider rating distributions
obtained from user studies, which are often times not available. In this paper
we extend these findings to show how to approximate user rating distributions
given a QoS-to-MOS mapping function and second order statistics. Such a user
rating distribution may then be combined with a QoS distribution observed in a
system to finally derive corresponding distributions of QoE scores. We provide
two examples to illustrate this process: 1) analytical results using a Web QoE
model relating waiting times to QoE, and 2) numerical results using
measurements relating packet losses to video stall pattern, which are in turn
mapped to QoE estimates. The results in this paper provide a solution to the
problem of understanding the QoE distribution in a system, in cases where the
necessary data is not directly available in the form of models going beyond the
MOS, or where the full details of subjective experiments are not available.Comment: 4th International Workshop on Quality of Experience Management (QoE
Management 2020), featured by IEEE Conference on Network Softwarization (IEEE
NetSoft 2020), Ghent, Belgiu
Impact of the Number of Votes on the Reliability and Validity of Subjective Speech Quality Assessment in the Crowdsourcing Approach
The subjective quality of transmitted speech is traditionally assessed in a
controlled laboratory environment according to ITU-T Rec. P.800. In turn, with
crowdsourcing, crowdworkers participate in a subjective online experiment using
their own listening device, and in their own working environment. Despite such
less controllable conditions, the increased use of crowdsourcing micro-task
platforms for quality assessment tasks has pushed a high demand for
standardized methods, resulting in ITU-T Rec. P.808. This work investigates the
impact of the number of judgments on the reliability and the validity of
quality ratings collected through crowdsourcing-based speech quality
assessments, as an input to ITU-T Rec. P.808 . Three crowdsourcing experiments
on different platforms were conducted to evaluate the overall quality of three
different speech datasets, using the Absolute Category Rating procedure. For
each dataset, the Mean Opinion Scores (MOS) are calculated using differing
numbers of crowdsourcing judgements. Then the results are compared to MOS
values collected in a standard laboratory experiment, to assess the validity of
crowdsourcing approach as a function of number of votes. In addition, the
reliability of the average scores is analyzed by checking inter-rater
reliability, gain in certainty, and the confidence of the MOS. The results
provide a suggestion on the required number of votes per condition, and allow
to model its impact on validity and reliability.Comment: This paper has been accepted for publication in the 2020 Twelfth
International Conference on Quality of Multimedia Experience (QoMEX
Survey of Web-based Crowdsourcing Frameworks for Subjective Quality Assessment
The popularity of the crowdsourcing for performing various tasks online increased significantly in the past few years. The low cost and flexibility of crowdsourcing, in particular, attracted researchers in the field of subjective multimedia evaluations and Quality of Experience (QoE). Since online assessment of multimedia content is challenging, several dedicated frameworks were created to aid in the designing of the tests, including the support of the testing methodologies like ACR, DCR, and PC, setting up the tasks, training sessions, screening of the subjects, and storage of the resulted data. In this paper, we focus on the web-based frameworks for multimedia quality assessments that support commonly used crowdsourcing platforms such as Amazon Mechanical Turk and Microworkers. We provide a detailed overview of the crowdsourcing frameworks and evaluate them to aid researchers in the field of QoE assessment in the selection of frameworks and crowdsourcing platforms that are adequate for their experiments
TOWARDS A COMPREHENSIVE FRAMEWORK FOR QOE AND USER BEHAVIOR MODELLING
ABSTRACT While the modeling of QoE has made significant advances over the last couple of years, currently existing models still lack an integration of user behavior aspects and user context factors along with the consideration of appropriate temporal scales. Therefore, the goal of this paper is to present a comprehensive QoE and user behavior model providing a framework which allows joining a multitude of existing modeling approaches under the perspectives of service provider benefit, user well-being and technical system performance. In addition, we discuss the role of a broad range of corresponding influence factors, with a specific emphasis on user and context issues, and illustrate our proposal through a series of related use cases
Pervasive Communities in the Internet of People
Peer reviewe